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(Dated: February 7, 2008) 

We present a search for the flavor-changing neutral current decay Bg -^ <j> fi'^ fi^ using about 
0.45 fb^'^ of data collected in pp collisions at y^=1.96 TeV with the D0 detector at the Fermilab 
Tevatron Collider. We find an upper limit on the branching ratio of this decay normalized to 



B 



3(Bg^0A.+ M~) 



< 4.4 X 10 ^ at the 95% C.L. Using the central value of the world 



^/V" 4> of B(a0^j/^^) 

average branching fraction of B^ — > J/-!/) <f), the limit corresponds to B{B1 
at the 95% C.L., the most stringent upper bound to date. 



)/.t+^") < 4.1 X 10" 



PACS numbers: 13.20.He, 12.15.Mm, 14.40.Nd 



The investigation of rare flavor changing neutral cur- 
rent (FCNC) B meson decays has received special atten- 
tion in the past since this opens up the possibility of pre- 
cision tests of the flavor structure of the standard model 
(SM). In the SM, FCNC decays are absent at tree level 
but proceed at higher order through electroweak pen- 
guin and box diagrams. FCNC decays are sensitive to 
new physics, since decay amplitudes involving new par- 
ticles interfere with SM amplitudes. Although inclusive 
FCNC decays like B -^ Xs(.+i- or B -^ X,j are the- 
oretically easier to calculate, exclusive decays with one 
hadron in the final state are experimentally easier to 
study. For instance, the exclusive decays B^ -^ K* i'^i~ 
and B^ -^ K^ £'^£~ have been already measured at B- 
factories .1,, 2] and were found to be consistent with the 
SM within the present experimental uncertainties. Re- 
lated to the same quark-level transition of 6 -^ s £^i^ is 
the corresponding exclusive FCNC decay B^ -^ (f) jjl^ n'' 
in the B'^ meson system. An observation of this decay or 
experimental upper limit on its rate will yield additional 
important information on the flavor dynamics of FCNC 
decays. 

Within the SM, the decay rate for the B^ -^ (pn'^ji^ 
decay, neglecting the interference effects with the much 
stronger 5° -^ J/'4'4' and 5° -^ ■0(25') (/) resonance de- 
cays, is predicted to be of the order of 1.6 x 10^^ 3] 
with about 30% uncertainty due to poorly known form 
factors. The interference effects with the B^ resonance 
decay amplitudes are large, with their expected magni- 
tude depending on the exact modeling of the charmo- 
nium states y. To separate experimentally the FCNC- 
mediated process i?^ -^ 0/x+/i^, one has to restrict the 
invariant mass of the final state lepton pair to be outside 
the charmonium resonances. Presently, the only existing 
experimental bound on the B^ —>■ (p ^^fi~ decay is given 
by CDF from the analysis of Run I data 5| . CDF sets 
an upper limit at the 95% C.L. of B{B^ -^ 4'fj.+^~) < 
6.7 X 10-^ 



In this Letter, we report on a new experimental limit 
on the decay _B^ — > ip ijl^ ^i~ , that is an order of magnitude 
more stringent than the existing limit. The <j) mesons 
are reconstructed through their K^ K^ decay mode and 
the invariant mass of the two muons in the final state is 
required to be outside the charmonium resonances. The 
events in our search are normalized to resonant decay 
B° — !■ J/-0 (f) events. Using the B^ -^ J/ip (j) mode as 
the normalization channel has the advantage that the 
efficiencies to detect the (f>fi^fj,~ system in signal and 
normalization events are similar, and systematic effects 



tend to cancel. 

The search uses a data set corresponding to approx- 
imately 0.45 fb~^of pp collisions at ^/s = 1.96 TeV 
recorded by the D0 detector operating at the Fermilab 
Tevatron Collider. The D0 detector is described in de- 
tail elsewhere [a|. The main elements relevant for this 
analysis are the central tracking and muon detector sys- 
tems. The central tracking system consists of a silicon mi- 
crostrip tracker (SMT) and a central fiber tracker (CFT), 
both located within a 2 T superconducting solenoidal 
magnet. The muon detector, which is located outside 
the calorimeter, consists of a layer of tracking detectors 
and scintillation trigger counters in front of 1.8 T toroidal 
magnets, followed by two more similar layers after the 
toroids, allowing for efficient muon detection out to pseu- 
dorapidity (r/) of ±2.0. 

Dimuon triggers were used in the data selection for this 
analysis. A trigger simulation was used to estimate the 
trigger efficiency for the signal and normalization sam- 
ples. These efficiencies were also checked with data sam- 
ples collected with single muon triggers. The event pre- 
selection starts with a loose selection of _B" -^ (f) fi^ ^^ 
candidates. These candidates are identified by requir- 
ing exactly two muons fulfilling quality cuts on the num- 
ber of hits in the muon system and the two additional 
charged particle tracks to form a good vertex. The re- 
constructed invariant mass of the B^ candidate should 
be within 4.4 < r7i^^+^- < 6.2 GcV/c^. 

We then require the invariant mass of the two muons 
to be within 0.5 < m^+^- < 4.4 GeV/c^. In this mass re- 
gion, the J/ip{-^ A^^M ) and 0(2S')(^ /^^A^^) resonances 
are excluded to discriminate against dominant resonant 
decays by rejecting the mass region 2.72 < m^+^- < 
4.06 GeV/c^. The J/4> mass resolution in data is given 
by a Gaussian distribution with cr = 75 MeV/c^ . The re- 
jected mass region then covers ±5a wide windows around 
the resonance masses. 

The x^ /d.o.f. of the two-muon vertex is required to 
be less than 10. The tracks that are matched to each 
muon are required to have at least three (four) measure- 
ments in the SMT (CFT) and the transverse momen- 
tum of each of the muons (p^) is required to be greater 
than 2.5 GeV/c with jr/j < 2.0 to be well inside the fidu- 
cial tracking and muon detector acceptances. In order 
to select well-measured secondary vertices, we define the 
two-dimensional decay length Lxy in the plane transverse 
to the beamline, and require its uncertainty 5Lxy to be 

less than 0.15 mm. L,,„ is calculated as L^,„ — "'"'J'^ , 



Jxy 



^xy 



where p|? is the transverse momentum of the candidate 



Sg , and lytx represents the vector pointing from the pri- 
mary vertex to the secondary vertex. The uncertainty on 
the transverse decay length, SL^y, is calculated by taking 
into account the uncertainties in both the primary and 
secondary vertex positions. The primary vertex itself is 
found for each event using a beam-spot constrained fit as 
described in Ref. [3. 

Next, the number of -8° -^ (pfj,'^fj.~ candidates is fur- 
ther reduced by requiring p^ > 5 GeV/c and asking the 
S" candidate vertex to have x^ < 36 with 5 d.o.f. The 
two tracks forming the 4> candidate are further required 
to have px > 0.7 GeV/c and their invariant mass within 
the range 1.008 < m^ < 1.032 GeV/c^. The successive 
cuts and the remaining candidates surviving each cut are 
shown in Tabled We apply the same selection for the 



TABLE I: Number of candidate events surviving the cuts in 
data used in the pre-selection analysis. 



Cut 


Value 


# candidates 


Good Bs vertex 




1555320 


Mass region (GeV/c^) 


0.5 < m^+^- <4.4 
excl. J/iP,iP{2S) 


530892 


Muon quality 




276875 


X^ /d.o.f . of vertex 


< 10 


127509 


Muon pt (GeV/c) 


> 2.5 


73555 


Muon r; 


< 2.0 


72350 


Tracking hits 


CFT> 3, SMT > 2 


58012 


SL^y (mm) 


< 0.15 


54752 


B° candidate pr (GeV/c) 


> 5.0 


54399 


B° x^ vertex 


< 36 


53195 


Kaon PT (GeV/c) 


> 0.7 


9639 


(j) mass (GeV/c^) 


1.008 <m<^ <1.032 


2602 



resonant B^ -^ J/ip 4> candidates except that the invari- 
ant mass of the muon pair is now required to be within 
±250 MeV/c2 of the J/V' mass. 

For the final event selection, we require the candidate 
events to satisfy additional criteria. The long lifetime 
of the 5° mesons allows us to reject the random combi- 
natoric background. For this purpose we use the decay 
length significance L^y/SL^y as one of the discriminating 
variables, since it gives better discriminating power than 
the transverse decay length alone. 

The fragmentation characteristics of the b quark are 
such that most of its momentum is carried by the B 
hadron. Thus the number of extra tracks near the S^ 
candidate tends to be small. Therefore the second dis- 
criminant is an isolation variable, X, of the muon and 
kaon pairs, defined as: 



I = 



\■p{(f)^l+^i )| 



|p((/>Ai+M )l 



E pi{An<i)- 

track i^B 



(1) 



Here, ^ pi is the scalar sum over all tracks exclud- 

track i^B 

ing the muon and kaon pairs within a cone of ATZ < 1 



around the momentum vector p{(f) ^~^ ^~) of the _B^ can- 
didate where ATZ = ^J{A(j)Y -\- (Ary)^. The final discrim- 
inating variable used is the pointing angle a, defined as 
the angle between the momentum vector p((/)/i+/i~) of 
the Bl candidate and the vector l^tx between the pri- 
mary and secondary vertices. This requirement ensures 
consistency between the direction of the decay vertex and 
the momentum vector of the B^ candidate. 

We generate signal Monte Carlo (MC) events for the 
decay B^ — > 0/i+/i~ using a decay model which includes 
the NNLO improved Wilson coefficients [g] for the short- 
distance part. The form factors obtained from QCD 
light-cone sum rules are taken from Ref. [2| . These form 
factors were originally determined for B —> K* tran- 
sitions and were compared with experimental measure- 
ments of the branching fraction B{B^ —* K* £'^£~) in 
Ref. p . Recently, new form factors for the B'^ —>■ 4> tran- 
sition, obtained from the light cone QCD sum rules, were 
published [li|. The difference between the form factors 
in Ref. '^ and those in Ref. [li| reaches 20% for m^+^- < 
1 GeV/c^, while elsewhere it remains well below 10%. 

The analysis is carried out based on signal MC events 
in the B'^ mass region and on data events in regions out- 
side the experimental signal window defined as 4.51 < 
"i</,M+M- < 6-13 GeV/c^. A 44 MeV/c^ mass shift in 
the mass region of interest is introduced to calibrate the 
D0 tracker. 

In order to avoid biasing the analysis procedure, data 
candidates in the signal mass region are not examined 
until completion of the analysis, and events in the side- 
band regions around the i?" mass are used instead. The 
expected mass resolution for 5° -^ cf) fx^ ^~ in the MC is 
75 MeV/c^. The start (end) of the upper (lower) side- 
band was chosen such that it is at least 270 MeV/c^ away 
from the B° mass. The widths of the sidebands used 
for background estimation are chosen to be 540 MeV/c^ 
each. The size of the blind signal region is ±225 MeV/c^ 
which corresponds to a ±3(t region around the -B^ mass. 
To determine the final limit on the branching fraction, 
we use a smaller mass re gion of ±2.5cr. 

A random-grid search |ll| was used to find simultane- 
ously the optimal values of the discriminants by maxi- 
mizing the figure of merit [IjI P = esig/{a/2 + y^Thmck). 
Here, esig is the reconstruction efficiency of the signal 
events relative to the preselection (estimated using MC), 
and iVback is the expected number of background events 
interpolated from the sidebands. The constant a is the 
number of standard deviations corresponding to the con- 
fidence level at which the signal hypothesis is tested. This 
constant a was set to 2.0, corresponding to about the 95% 
C.L. After optimization, we find the following values for 
the discriminating variables: L^y/SL^y > 10.3, X > 0.72, 
and a < 0.1 rad. 

The total signal efficiency relative to pre-selection of 
the three discriminating cuts is (54 ± 3)% where the un- 
certainty is statistical only. After a linear interpolation of 



the sideband population for the whole data sample into 
the mass window signal region, we obtain an expected 
number of 1.6±0.4 background events with statistical un- 
certainty only. 

Upon examining the data in the mass region, zero can- 
didate events are observed in the signal region, consistent 
with the background events as estimated from sidebands. 
Figure^shows the remaining events populating the lower 
and upper sidebands. The Poisson probability of observ- 
ing zero events for an expected background of 1.6 ± 0.4 
is p = 0.22. 



gion J/ ■)/;(— > fi^ ^i~)(j) is negligible. We therefore con- 
strain the two muons to have an invariant mass equal 
to the J/f/j mass [13 when calculating the fj,'^ii~K^K^ 
invariant mass. The mass spectrum of the reconstructed 
B^ — *■ J/tp (j) is shown in Fig. |21 A fit using a Gaussian 
function for the signal and a second order polynomial for 
the background yields 73 ± 10 ± 4 i?° candidates, where 
the first uncertainty is due to statistics and the second 
represents the systematic uncertainty which is estimated 
by varying the fit range as well as the background and 
signal shape hypotheses. 



r, 2 
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FIG. 1: The invariant mass distribution after optimized re- 
quirements on tlie discriminating variables. The solid line 
shows the sidebands background interpolation into the signal 
region. 

In the absence of an apparent signal, a limit on the 
branching fraction B{B'^ -^ (j) ji^ ii~) can be computed 
by normalizing the upper limit on the number of events 
in the -B^ signal region to the number of reconstructed 
B^ -^ J /ip (p events: 



B{B': 



'M M 



B{B^^J/^(I>) 



Nbo 






■ B{J/i, 



M> 



(2) 



where iVui is the upper limit on the number of signal de- 
cays, estimated from the number of observed events and 
expected background events, and N^o is the observed 
number of B^ -^ J /ip (p events. The measured branching 
fractions arc B{J/ip -> ^J,+^J,~) = (5.88 ± 0.10) x 10"^ 
and B{B'^ -^ J/iP(P) = (9.3 ±3.3) x 10"" ^. The global 
efficiencies of the signal and normalization channels are 
£0At+/j- 8'^d. ej/^0 respectively, and include all event se- 
lection cuts and the acceptance relative to the entire di- 
muon mass region. They are determined from MC yield- 
ing an efficiency ratio of (ej/^^/e^^+p-) = 2.80 ± 0.21, 
where the uncertainty is due to MC statistics. Applying 
no cut around the charmonium resonances the efficiency 
ratio would be (fj/v^/^lu+u-) ~ 1-06 ± 0.07. In order 
to avoid large uncertainties associated with the poorly 
known branching fraction of B'^ — > J/ip (p, we normalize 
the limit of B^ — > (/)/i+/i~ relative to /B(i3° -^ J/ipcp) as 
shown by Eq. El 

The same cuts are applied to the S^ -^ J/ip (p candi- 
dates. The contamination of muon pairs from the non- 
resonant (p fj:'^ fi~ decay in the resonant normalization re- 



FIG. 2: The normalization channel B^ —^ J/ip <p. 

The different sources of relative uncertainty that enter 
into the limit calculation of B of B'^ ^ (p fj,'^fj,~ are given 
in Table ITU The largest uncertainty, 25%, is due to the 
background interpolation into the signal region and is 
based on the statistical uncertainty of the fit integral. 
The uncertainty on the number of observed i?^ — > J/ip (p 
events in the normalization channel is 14.8%. 

The pt distribution of the B° in data is on aver- 
age slightly higher than that from MC. Therefore, MC 
events for the signal and normalization channels have 
been reweighted accordingly and an additional uncer- 
tainty of 3.7% is applied. The CP-even signal MC events 
are generated with a -8° lifetime of 1.44 ps [lj|. To 
account for a possible efficiency difference related with 
the shorter lifetime of the CP-even i3°, the signal MC 
events are weighted according to the combined world av- 
erage CP-even lifetime |l5J- The efficiency difference is 
estimated to be 8% which is taken as an additional sys- 
tematic uncertainty. The statistical uncertainty on the 
efficiency ratio ej/^^/e^/x+^i- is found to be 7.5%. The 
signal efficiency obtained from MC is based on the in- 
put for the NNLO Wilson coefficients and form factors 
of Ref . '^ . We do not include any theoretical uncertainty 
in our systematics uncertainty estimation. 

The statistical and systematic uncertainties can be in- 
cluded in the limit calculation by integrating over proba- 
bility functions that parameterize the uncertainties. We 
use a prescription [1^1 where we construct a frequentist 
confidence interval with the Feldman and Cousins |l7l | or- 
dering scheme for the MC integration. The background 



TABLE II: The relative uncertainties found for the upper 
limit on B. 



Source 



# of B]; -^ j/v<^ 

MC weighting 
CP-even lifetime 
B{J/ij ^ mm) 



Total 



Relative Uncertainty [%] 



Background uncertainty 



14.8 
7.5 
3.7 
8.0 
1.7 



18.9 



25.0 
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is modeled as a Gaussian distribution with its mean value 
equal to the expected number of background events and 
its standard deviation equal to the background uncer- 
tainty. Including the statistical and systematic uncer- 
tainties, the Feldman and Cousins (FC) limit is 






< 4.4 (3.5) X 10 



-3 



at the 95% (90%) C.L. respectively. Taking a Bayesian 
approach J^j with a flat prior and the uncertainties 
treated as Gaussian distributions in the integration, 
we find an upper hmit of B{B^ -^ (f) ^'^ ijl~)/B{B^ -^ 
J/^pcj)) < 7.4(5.6) X 10-3 at the 95% (90%) C.L., respec- 
tively. 

Since we have fewer events observed than expected, we 
also quote the sensitivity of our search. Assuming there 
is only background, we calculate for each possible value of 
observation a 95% C.L. upper limit weighted by the Pois- 
son probability of occurrence. Including the statistical 
and systematical uncertainties, our sensitivity is given by 
{B{B° -^ 4>^i+^I-))/B{B° -^ J/iP(l)) = 1.1(1.2) X 10-2 
at the 95% C.L. using the FC (Bayesian) approaches, 
respectively. 

Using only the central value of the world average 
branching fraction J^J oiB{B'^ ~* J/V"/-) = (9.3±3.3) x 
lO"'', the FC limit corresponds to B{B° -^ (t> ^J.'^ ^J.^ ) < 
4.1 (3.2) X 10-6 at the 95% (90%) C.L. respectively This 
is presently the most stringent upper bound and can be 
compared with the SM calculation of B{B^ ^ cpfi'^i 
1.6 X 10-6 of Ref. ^. 
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